COMMENT ⊗   VALID 00003 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002		Purger Documentation
C00014 00003	*19 Sep 1981	disk allocations in blocks (tracks)	ME
C00022 ENDMK
C⊗;
	Purger Documentation

	R. E. Gorin, June 7, 1972 (updated from time to time since then)


	The purger program is used to enforce the cost recovery policies
of CSD Computer Facilities as they pertain to the SAIL system.  Each user
has a disk space allocation; if you are over your allocation, the purger
will delete enough of your files (in an order you can specify) to bring
your usage down to or below your allocation.

	NOTICE:  A disk purge may occur at any time (with the system up or
while it is down) without any warning.  However, a purge is UNLIKELY when
there are over 1000 free disk blocks, and very LIKELY when there are less
than 400 free blocks (note that LOGIN gives a warning whenever disk space
is low when you log in).  Although the purger may occasionally give a
purge warning (particularly to people who are over allocation), such
warnings will not always occur.  You are responsible for keeping track of
your own disk usage, if you want to avoid the purger.

	If you need a permanently larger disk quota, MAIL a message to Susan
Hill (SH at SAIL or CSD.HILL at SCORE) explaining how large a quota you
want and who'll pay for it.

	If you need to be temporarily spared from the purger because some
critical files are temporarily pushing you over your quota, MAIL a note to
PURGER at SAIL explaining why you should be spared and for how long you
expect to be over your quota; the purger will consider your request.  It
may be that temporary increases in the amount of disk space that you
consume will be reflected as increased charges for your computer account.

	The purger counts the size of each user's files.  This also
includes files on [2,2] that belong to the user, such as mail and
plan files.  If a user is known to the system management then there
will be a disk quota associated with that user.  Unknown users have a
disk quota of zero.

	If a user is over his allocation then the purger will select
files to delete.  To find out if you are over your allocation, use
the monitor command DSKSIZ.  For documentation on the DSKSIZ program,
type the monitor command HELP DSKSIZ (or READ DSKSIZ from a display).

	A user's disk quota is expressed in blocks, each one of which
corresponds to 1 track on the disk.  Each file uses up an integral number
of blocks, although an average file may not utilize half of the data space
in the file's last block.  See page 3 of this file for some details on
disk allocations in blocks.

	An authorized user who is over his quota can specify the
purge order for his files.  This specification is accomplished by a
line in the file OPTION.TXT on any of the user's areas.  The line
that the purger reads is called the "purge line." Only one purge line
is allowed for each user. The purger will read each of the user's
OPTION.TXT files (since there may be one for each of his disk areas)
until it finds one with a purge line. (Note that if two option files
have purge lines only one of these will be seen. The order in which
option files are read is undefined.)

	The "purge line" has the following format.  In this description
curly brackets "{" and "}" denote optional items.  Pointy brackets "<"
and ">" denote syntactic quantities which are defined below. The
asterisk that appears below is the Kleene star.

<Purge Line>	→	PURGE: <term>{,<term>}*;
<term> 		→	{<purge order>,}{<proj>}<file spec>
<file spec>	→	<name>{.<name>}{<proj>}
<proj>		→	[<name>]
<purge order>	→	#<number>

<number>	→	'string of decimal digits'
<name>		→	'string of alpha-numeric characters' |
			↓'string of sixbit characters'↓  |
			*

strings above that  are enclosed  in single  quotes are  meant to  be
informal definitions of the data required.

Semantics applying to the purge line.

	Each of a user's files is assigned a number, the purge order,
that determines the order of file deletion.  The purge line defines
the purge order of every one of a user's files.

	Every <term> in the purge line has a purge order associated
with it. A term that does not explicitly specify a <purge order> will
have a purge order that is one greater than the term to its left.
The first term on the line (the leftmost) will have purge order = 1,
unless that is over-ridden by an explicit purge order.  If a <purge
order> term is seen then that purge order will be associated with the
next term.  Subsequent terms will be given consecutive purge orders
unless reset by another <purge order>.

	If [<proj>] appears at the front of a term, then that term
and all subsequent terms will apply to that project area until reset
by another such term.


	The terms are then sorted in order of their "degree of
specification." The "degree of specification" is defined as follows:

	Term 		degree of specification

	nnnnnn.eee[ppp]		1		most specific
	nnnnnn.*  [ppp]		2
	*     .eee[ppp]		3
	*     .*  [ppp]		4
	nnnnnn.eee[*]		5
	nnnnnn.*  [*]		6
	*     .eee[*]		7
	*     .*  [*]		8		least specific

	The most specific terms are processed first.  Any file
matched by a term, and not matched by a more specific term, is given
that term's purge order.  When all terms are exhausted, any files
that have not been matched are given a purge order that is greater
than any purge order used thus far.

	Files are deleted in increasing purge order until the user is
under allocation.  The purger will not delete any files on [2,2]
belonging to the user even though it counts them against the user's
allocation.  If the last file deleted is large, the user may wind up
way under allocation.  This is not a malfunction in the purger, but
rather in the user.

	A purge may create a DART P-dump of all deleted files.  If
this is the case, a file that is a candidate for purging need not
have ever been dumped before.  Otherwise, no files belonging to an
authorized user will be deleted unless they have been P-dumped or are
marked dump never.


	If no purge line is found in any option file then the
following default line will be used:

PURGE:*.TMP,*.TEM,*.RPG,*.LST,*.REL,*.BIN,*.VRN,*.DMP,*.LAP,*.XGP,*.PRE;

NOTE:     If the purge line in an OPTION.TXT file does not
	  completely specify the purge order of ALL the
	  user's files, then the remaining files will all be
	  given the SAME purge order.  The purger will NOT
	  automatically apply the default purge order to
	  the remaining files.  It is up to the user to have
	  a complete purge line.  The default purge order
	  applies only when there is NO purge line given in
	  an OPTION.TXT file.  You may want to copy the
	  default purge order above into some part of your
	  purge line in OPTION.TXT.


	The OPTION.TXT file may be in E or SOS format or may be  bare
text.


Example:

PURGE:	FOO.*,GARPLY.REL[S],*.REL,*.RPG,*.TMP,*.*,FOO.BAZ,GARPLY.*,*.*[s];
This example is fairly complex.  
	Term		Purge order		Specification
	FOO.*		1			6
	GARPLY.REL[S]	2			1
	*.REL		3			7
	*.RPG		4			7
	*.TMP		5			7
	*.*		6			8
	FOO.BAZ		7			5
	GARPLY.*	8			6
	*.*[s]		9			4

Files will be purged in the following order:
	1.	FOO.*[*], except see 7 and 9 below.
	2.	Garply.rel[s] 
	3,4,5	*.rel,*.rpg,*.tmp (except see 8 below)
	6	any other files, except see below
	7	Foo.baz[*]
	8	Garply.*[*]
	9	*.*[s]

	foo.* will not delete foo.baz[*], or foo.*[s]
	Garply.rel[s] will be not be given order 8 or 9 because
		it is more specific.
	rel, rpg and tmp files will not be deleted from [s] until 9.
	Any files not described by 7,8,9 will be deleted by 6.
*19 Sep 1981	disk allocations in blocks (tracks)	ME
SAIL disk allocations and disk usage are now measured in blocks used on
the disk, instead of in data words.  A block is the smallest unit of disk
space that can be incrementally assigned to a file and is currently (and
has been for many years) one track (2.25K data words).  Allocation by
block usage means that the charge for a file is directly and precisely
proportional to the amount of free disk space the file uses up.
Therefore, each small file (which is wasteful of data space on the disk)
will now use up a higher proportion of a user's allocation, since the
minimum size of a file is one block.  In the extreme example, an empty
file will now be charged as 1 block (which it is) although before it was
counted as 0 data words.  Even the average non-small file will now be
charged for 1/2 block more of data space, since such an average file ends
in the middle of the last block it uses.  Because of these facts, many
users may find that they are now over allocation although they were under
before.  The allocation unit, namely one aliquot, is now 50 blocks; all
allocations have been converted to the new system, maintaining the aliquot
count of each user.  Users interested in buying more aliquots should
contact Susan Hill (SH at SAIL).

(Let me point out that we expect that anyone who was at allocation before
to be over now.  And we expect that such people will have to buy more
aliquots and therefore pay "more" for their disk space than they did
"before".  However, let me point out also that SAIL "lost money" last year
because the disk space was being sold for less than it should have been,
because of a couple of logical errors in the price computation.  Therefore
the already announced disk prices for September were planned back in early
summer to represent an increase, as was mentioned with the announcement of
the intermediate July and August prices.  Actually, the "smaller" aliquot
(50 blocks, instead of, say, 100) now available represents a finer
resolution in the amount of disk space a user can request; in particular,
it allows a user who wants the minimum amount of space to pay less than
he would have to if the aliquot size were bigger.  Similarly, large disk
users can specify disk requirements more precisely with a smaller minimum
unit (i.e., a smaller aliquot).  The main problem, of course, is that
many people will have now to request adjustments in the number of aliquots
they buy.)

An alternative to increasing your allocation (if you are now over quota),
is to compress or combine your files.  For example, if you have many small
DO files, you can put them into one DO file with labels on the individual
parts; thus you would say DO XX(FOO) instead of DO FOO; for details, say
READ DO.  There are also (old) file combining and data compression
programs available, such as BUNDLE, SPINDL, etc.  See the individual
program's documentation, usually findable with the READ <program-name>
command.

The DSKSIZ monitor command now checks disk usage in blocks, using the new
allocations.

Because of the new allocation scheme, the output format of the DIRECTORY
monitor command has been changed to show each file's size in blocks
instead of data words and K data words.  To provide information on both
data size and block count for each file, DIRECTORY shows the file size in
tenths of a block, rounded up (ceiling rounding).  Thus if a file is
listed as 2.4 (blocks), then the user will be charged for 3 blocks for
that file, although at least 3/5 of the last block is empty.  A file size
of 3.0 means the user will be charged for 3 blocks and the last block is
approximately full.  Although an empty file will be shown with a file size
of 0.0, it will nevertheless be charged for as 1 block.  The totals
printed by DIRECTORY show the total block charges for the files being
totalled (counting each file as an integral number of blocks).

Although DIRECTORY now gives file sizes in blocks, it is possible to also
get the formerly-listed file data size, in words or K words, shown under a
heading of "Words".  This is achieved by using the /WORDS switch in the
DIRECTORY command, e.g., DI FOO/WORDS or DI/WORDS FOO.  This switch is
always global in effect, no matter where it appears in the command, like
all the other DIRECTORY information-requesting switches.

The DIRED and WILD programs, which also show file size information, have
NOT been changed to display block usage.
[WILD has been changed -- JJW 26-Sep-81]

Any complaints about the new allocation scheme should be sent to Ralph
Gorin (REG at SAIL).